Determining color at an object point from multple images providing conflicting color information

ABSTRACT

A set of images of an object are captured from a set of lenses positioned at different angles. A common point is determined, which has a color value that differs among the images. Color at the point is mathematically expressed as a set of equations. The equations comprise a recorded color value for the point, an ambient contribution, a diffuse contribution, and a specular contribution. The ambient and diffuse contributions are set as equal across the equations. The specular contribution is determined, such as by quantifying the light sources and solving for the specular contribution using a Phong lighting model equation. True color for the point, which is based on the ambient and diffuse contribution, is determined by solving a set of simultaneous equations once the specular contribution is known. An image is created where the point has the true color as determined above.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/145,023, filed on Dec. 31, 2013 of which is incorporated herein inits entirety.

BACKGROUND

The present invention relates to the field of color reproduction and,more particularly, to determining color at an object point from multipleimages providing conflicting color information.

Cameras with multiple fixed lenses are increasingly being utilized tocapture imagery in an arc which is greater than what a single lens isable to provide. For any given point, however, the different cameralenses record different color values. These differences are primarily aresult of specular reflection. Semi-reflective surfaces are ofparticular concern for color accuracy. For example, if a redsemi-reflective surface is fifty percent diffuse and fifty percentreflective and a light source is white, then color of the surface ascaptured by a camera lens will appear from red to white color dependingon viewing angle and the intensity of the white light source.

Differences in color from different images must be resolved if theimages are to be stitched together to form a cohesive scheme. Whencombining content from different images into a single one, conventionaltechniques generally try to smooth the color differences, which resultsin a cohesive combined image. The resulting image, however, does notaccurately represent the true color of the object being captured. Forsemi-reflective surfaces, color accuracy resulting from conventionaltechniques can be particularly poor. Stated differently, conventionaltechniques for combining points captured by multiple lenses fail toproduce photorealistic images. Which is a problem heretofore unresolved.

Additionally, many image analysis techniques assume an object's truecolor is represented. These techniques yield inaccurate results whenthis assumption is incorrect. For instance, color of an image is oftenused to determine a shape of a reconstructed surface. The more accuratethe color; the more accurate the determined shape. Embodiments of thedisclosure ensure true color of surfaces is maintained when new imagesare being generated by stitching together a set of images captured usingdifferent lenses positioned at different angles relative to a surface.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 shows a flowchart for determining accurate color ofsemi-reflective surfaces captured from multiple different camera lensesand angles in accordance with embodiments of the disclosure.

FIG. 2A shows a diagram for an illumination calculation in accordancewith an embodiment of the disclosure.

FIG. 2B shows a diagram for an illumination calculation for diffuselight in accordance with an embodiment of the disclosure.

FIG. 2C shows a diagram for an illumination calculation for ambientlight in accordance with an embodiment of the disclosure.

FIG. 2D shows a diagram for an illumination calculation for Specularlight in accordance with an embodiment of the disclosure.

FIG. 3 illustrates a scenario for determining true color of asemi-reflective surface in accordance with an embodiment of theinventive arrangements disclosed herein.

FIG. 4 shows a diagram of a system for determining a true color ofsemi-reflective surface in accordance with an embodiment of theinventive arrangements disclosed herein.

DETAILED DESCRIPTION

The disclosure presents an innovation for improving color accuracy ofimages. The color improvements result from using lenses at differentangles to capture an image of a point. These different angles forviewing a common point on a surface may result in different capturedcolors, when the surface is a semi-reflective one. One application ofthe innovation is to improve color realism for images created bycombining multiple images from multiple different lenses. The disclosedtechnique greatly improves color accuracy for semi-reflective surfaces,which are especially sensitive to different angles.

To elaborate, assume that each camera lens captures light havingambient, diffuse, and specular contributions. For a point (vertex),final illumination can equal ambient+diffuse+specular contributions. Forchanges in camera angles, the ambient and diffuse contributions forcolor of a camera captured image are relatively insensitive to angle.That means that for a set of ideal camera lenses capturing a color at apoint of a surface, the ambient and diffuse contributions to color willbe equal (at least in regard to recorded color at a surface point)across camera lenses regardless of camera angle. Thus, any deviations incolor from different lenses of a common scene for a point or set ofpoints are primarily a result of specular contributions (which aresensitive to angle).

The disclosure calculates the specular contributions for a vertex (orset of points) for each image captured from one of a set of differentlenses. These specular contributions are mathematically removed from theequation, diffuse and ambient contributions are retained. This diffuseand ambient contribution combination represents a true color of thevertex or set of points. Not all embodiments of the disclosed inventionneed utilize both ambient and diffuse contributions, and derivativeembodiments (where true color is based solely on a diffuse contributionfor example) are contemplated and are to be considered within scope ofthe disclosure.

When generating an image that stitches together content from a set ofimages, color of the generated image is based on this computed truecolor. Inconsistencies in true color calculations across multiple imagesfrom different lenses will be minor compared to the deviations resultingfrom the specular contributions, which have been removed. Thus, thedisclosed technique results in colors of combined images being moreaccurate and precise than would be otherwise possible. Photorealisticcolor within a photograph is therefore possible, though none of theindividual images from individual camera lenses provides images having aphotorealistic color.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method, or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Turning to the Figures, FIG. 1 shows a flowchart for determiningaccurate color of semi-reflective surfaces captured from multipledifferent camera lenses and angles in accordance with embodiments of thedisclosure. In step 105, a system can be established where multipledifferent camera lenses capture images of a surface. In one embodiment,the multiple different camera lenses can be fixed lenses of a singlecamera device. These fixed lenses can cover a same relative region(using stereoscopic imagery techniques for depth capture for threedimensional images) or can cover different environmental regions, wheresome overlap between the different regions exists (and where colordifferences will exist for the same point within the regions ofoverlap). In another embodiment, different cameras can be utilized tocapture the images. Any arrangement may be used so long as significantlydifferent angles are available to lenses capturing a point on a surfaceof an object.

In step 110, a volumetric space can be defined for computationalpurposes. This volumetric space can include the object/surface beingcaptured by two or more lenses, the lenses themselves, and a set of oneor more light sources. Relative positions and angles of the lenses,light sources, and surface are defined for the volumetric space. Thevolumetric space models relative positions of objects and angles, wherethe volumetric space includes all relevant data needed for illuminationmodel calculations, as described herein. In one embodiment, positionsfor the light sources may be initially unknown. When this is the case,these positions can be calculated, as shown by step 115. If the positionand intensity of the light source(s) are known (possibly by usingenvironmental sensors) calculations detailed for embodiments of step 115are unnecessary.

In step 115, for each light source, a position, color, and intensity forthe light source can be defined. Quantifying these light sourcevariables can be performed in numerous manners, such as using directmeasured values from light sensing equipment, using calculations basedon imagery, and combinations thereof. Assuming light sources arequantified by calculations, these calculations may utilize images fromthe multiple lenses by taking multiple different points common amongimages from different lenses. Known calculation techniques can beapplied at this step, which generally utilize linear equations withuncertainty to calculate a number of light sources, light sourcepositions, light source colors, and light source intensities for alllight sources significantly affecting the volumetric space. For example,Planckian locus or black body locus can be used for these calculations,as detailed in numerous publications such as the document titled “Methodand Measuring and Specifying Colour Rendering Properties of LightSources” having ISBN 9783900734572.

Once light sources have been quantified, color information for a set ofone or more surface points or vertex can be identified, as shown by step120. The same point (or set of points) is captured by multiple lenses atthis step. Color the point (or set of points) as captured by eachdifferent lens will have appreciable deviations, at this stage, which isnoted by step 125.

It can be assumed that the point of color for each image is able to berepresented using a mathematical equation with variable values fordiffuse, ambient, and specular contributions. So if C1, C2, and C3represent color values for the same point from different cameras, thefollowing equations apply:C ₁ =A ₁ (Ambient)+D ₁ (Diffuse)+S ₁ (Specular)C ₂ =A ₂ +D ₂ +S ₂C ₃ =A ₃ +D ₃ +S ₃

It should be understood that the above simple models for color are notthe only equations that are able to be utilized, and that the disclosurecontemplates use of other equations having similar effect.

Since ambient and diffuse contributions are relatively insensitive toangle, these contributions are approximately equivalent regardless ofthe lens A₁=A₂=A₃; and D₁=D₂=D₃; and (A₁+D₁)=(A₂+D₂)=(A₃+D₃). Of coursethis assumption is an approximation for ideal surfaces. Optic componentand real world abnormalities will be present and should be considered,which is why the above statement is that these values are onlyapproximately equivalent. The above equations can be rewritten (forsimplicity of expressions, hereafter) using a common variable for theambient and diffuse componentsC ₁ =AD+S ₁C ₂ =AD+S ₂C ₃ =AD+S ₃

The value for C₁, C₂, and C₃ are known, since they are defined withinthe captured images. Values for AD, S₁, S₂, and S₃ are unknown at thisstage. Without more, there are three equations with four unknowns.However, because the light source specifics were quantified in step 115and because the position and angles of the camera lenses (from step 110)are known, values of S₁, S₂, and S₃ are able to be calculated, as shownby step 130. The specular color contribution (S₁, S₂, and S₃) for apoint in an image equals the sum of the specular color contributionsfrom each light source. A Phong lighting model (or other computation)can be utilized for these calculations (See step 135).specular=Ks*I*cos^(n)(φ)

-   -   Ks: specular reflection coefficient    -   I: light intensity    -   φ: angle between camera lens and a vertex (twice the angle        differences between the light source and the surface normal)

Once the values for specular components are calculated, equations andvalues exist for calculating AD, which represents the true color of theone or more points on the surface. After calculating the Specularcomponent, where are X number of equations and X-1 number of unknowns,so a simultaneous equation is able to be solved for the variables (withan extra “equation” existing so that overall inaccuracies can beminimized). Step 140 indicates that the simultaneous solution for truecolor is calculated. In step 144, if there are additional points on thesurface for which color is to be calculated then the process can repeat,which is shown by progressing from step 145 to step 120. Otherwise, theprocess can end in step 150.

It should be emphasized from the example above, that solving for eachspecular coefficient for each equation results in an “extra” value,which is strictly speaking unnecessary to solve for the AD value. Thisextra value can be used to minimize errors in the calculations andunderlying models. This minimization can be utilized in various manners,such as being able to disregard a value/equation should that value bestatistically deviant from other ones of the equations. There are manycontextual reasons for this, such as having one of the videosimprecisely match the surface point when mapping a resulting image tothe vertex. In other situations, one or more of the light sources may betemporarily blocked (by a physical obstruction), which doesn't affectother ones of the lens captured images. In another example, a relativelyextreme angle for a single one of the camera lenses can have adisproportionate effect on calculation accuracy. All of these deviationscan be statistically accounted for utilizing a variety of techniquesenabled, at least in part by, having extra data for the mathematicalmodels to improve accuracy and to minimize calculation abnormalitiesthat would otherwise impact accuracy of the color determination.

Additionally, disclosure embodiments contemplate leveraging informationfrom past calculations to ensure accuracy of future calculations. Forexample, if the lenses of a camera capture video, different time periodsof the video can be used to calculate true color for the same surfacepoint. For example, five different time frames can be sampled, wherevalues from an average of the two “best” color values for a given vertexare used in calculating the true color of the point. This type of errormitigation can be especially useful for video captured in extremeenvironment subject to significant deviations. An example would be foran omnidirectional camera with fixed lenses attached to a movingvehicle—as lighting conditions, obstructions, movement, and otherenvironmental factors are anticipated in such a context.

FIG. 2A shows a diagram for an illumination calculation in accordancewith an embodiment of the disclosure. The diagram shows a light source210, which illuminates a point 215 on a surface 220. Camera 225 capturesan image that includes point 215. The illumination of point 215 capturedby camera 225 can be expressed as a diffuse component plus an ambientcomponent plus a specular component.

FIG. 2B is an illustration for a diffuse component for the point 215.Diffuse light is an illumination that a surface receives from a lightsource that reflects equally in all directions. It does not matter wherea camera is positioned relatively to the angle of the light, since thediffuse light component is angle insensitive with regards to color.Lamberts law (or other such equation) can be used to determine how muchlight from the light source 210 is received by the point. Differentobjects can receive different amounts of diffuse light. The amount ofdiffuse light that is captured by the camera can be described asequation 235. In question 235, K_(d) is a reflection coefficient; I isan intensity, and the angle (θ) is a vector from the object to the lightsource.

FIG. 2C shows a diagram for an illumination calculation for ambientlight in accordance with an embodiment of the disclosure. Ambient lightor background light is light that is scattered by the environment. Thatis, a set of objects 240 in an environment reflect light, whichoriginates from a light source 210 and is captured by camera 225.Objects resulting in ambient light that is captured by the camera 225may include walls, floors, ceilings, and other reflective andsemi-reflective objects. Ambient light is independent of light position,object orientation, camera position or orientation. Ambient light has nodirection. Radiosity is the calculation applicable for radiant light.The amount of ambient light that can be seen from an object isdetermined from equation 245. Light source 210 has an ambient lightcontribution of I. Different objects in an environment (like walls of aroom) can reflect different amounts of ambient, which is defined by anambient reflection coefficient K_(a).

FIG. 2D shows a diagram for an illumination calculation for Specularlight in accordance with an embodiment of the disclosure. Specular lightis a minor-like reflection of light from a surface, in which light froma single incoming direction (a ray) is reflected into a single outgoingdirection. Specular light can be shown as a bright spot on an object.The amount of specular light is highly dependent on an angle of rays oflight from the light source 210 being reflected off the point 215towards the camera 225. The specular component of light is sensitive toangle. Equation 250 shows an equation for calculating the specularcomponent.

Most real world objects have some mixture of diffuse and specularreflective properties. The specular component (of a surface havingspecular reflective properties), reflectivity is nearly zero at allangles except at the appropriate reflective angle. That is, reflectedradiation will follow a different path from incident radiation for allcases other than radiation normal to the surface. For the diffusecomponent (of a surface having a diffuse reflective properties),reflectivity is uniform so radiation is reflected in all angles equallyor near equally. Diffusely reflecting surfaces (the diffuse component)has a property referred to as Labertian reflectance, which means thatapparent brightness is the same regardless of the observer's angle ofview. When a colored object has both diffuse and specular reflectionusually only the diffuse component is colored. If a diffuse surface iscolored, the reflected light is also colored, resulting in similarcoloration of surrounding objects.

For purposes of this disclosure, a true color of a surface at a point isthe color resulting from diffuse and ambient light. Color changes fromthis baseline resulting from the specular component are deviations fromthe true color, which is why the disclosure minimizes/eliminates thecoloration changes present in the images that result from specularreflections.

FIG. 3 illustrates a scenario 310 for determining true color of asemi-reflective surface in accordance with an embodiment of theinventive arrangements disclosed herein. Multiple cameras 301, 302, 303,304, 305 are shown, which capture images 361, 362, 363, 364, 365 thatshow point 330 illuminated by light source 321 and 322. Each of theimages 361 can indicate different colors for the point 330. A colorengine 311 can process the images 361-365 to determine a true color ofthe point 330, as detailed herein. The color engine 311 can determinetrue color for multiple points of a semi-reflective surface 340 ofobject 342 to increase the color accuracy of the surface 340 as shown ina captured set of images or images derived from a captured set ofimages. In one embodiment, the determination of true color can be partof a process of stitching together content of the images 361-365 into asingle image, where the resulting image has a more realistic color forthe surface 340 than any of the component images 361-365.

Color engine 311 can utilize images 361-365 from cameras 301-305 todetermine the position of the point 330 as recorded within each image361-365 (e.g., or video frame). Each of the points (including point 330)within the images 361-165 can be associated with a color (e.g. color C).Each color (e.g., color C) can be a combination of true color (e.g., DA)and specular contributions. In other words, specular contributions for apoint's image color can be sensitive to camera 301-305 angle, whilediffuse and/or ambient contributions for the point's coordinates areinsensitive to camera 301-305 angle. The processing of a set of images361-165 performed by color engine 311 can be consistent with the processillustrated by FIG. 1, in one contemplated embodiment of the disclosure.Fewer or greater numbers of cameras 301-305 and light sources 321, 322are contemplated. In one embodiment of the disclosure, a single camera(referred to as an omnidirectional camera or a three hundred and sixtydegree camera) with multiple fixed lenses can be used, where each lensof the omnidirectional camera is equivalent to one of the shown cameras301-305.

In scenario 310, a semi-reflective surface can be a portion of an object342 within a real world environment 352. The object 342 can correspondto any real-world object including, but not limited to, a ball, a table,a wall, a vehicle, and the like. For example, object 342 can be a devicehaving a semi-reflective surface 340 such as a display. Asemi-reflective surface has an appreciable mixture of diffuse andspecular reflective properties—such that a color value captured by oneor more cameras 301-305 for the surface 340 varies appreciably based onangles involved.

The environment 352 can correspond to a three dimensional environmentwhich can be mapped to one or more three dimensional coordinate systems(e.g., coordinate system 350). For example, the environment 352, cameras301, light source 321, 322, object 342, surface 340, and surface point330 can each be mapped to a coordinate triplet (e.g., x,y,z). Thevolumetric space of FIG. 3 models the real world environment 352.

As used herein, specular reflection can be the minor-like reflection oflight (e.g., light ray 332) from a surface (e.g., surface 340), in whichlight from a single incoming direction (e.g., ray 332) can be reflectedinto a single outgoing direction (e.g., reflection 334). The angle ofthe reflection 334 depends on the surface normal 336. Specularreflection can be distinct from diffuse reflection, where incoming lightis reflected in a broad range of directions. For example, if red surfaceis 50% diffuse and 50% specular and light source is white, then color ofsurface will appear from red to white color depending on viewing angleand intensity of light source.

Cameras 301-305 can be one or more optical instruments which can recordimages (e.g., digital encoding) and stored directly or transmitted toanother location. Cameras 301-305 can capture images 361-365 of one ormore portions of surface 340 (e.g., point 330) which can be conveyed toengine 311. It should be appreciated that cameras 301-305 can observedifferent colors at point 330 due to the different positions. That is,some cameras 301-305 can receive more or less specular reflection 334from light sources 321, 322 resulting in color variations of point 330within images 361-365.

Light sources 321, 322 can be one or more thermal bodies that, at agiven temperature, emit a characteristic spectrum of black-bodyradiation. Light source 321, 322 can include, but is not limited to, alight bulb, the sun, a flame, and the like. Source 321, 322 cancorrespond to a spot light, an ambient light, and the like. It should beappreciated that source 321, 322 can emit a visible wavelength (e.g.,white) or a portion of a visible wavelength (e.g., red, blue, green).

It should be appreciated that environment 352 can be a static or dynamicenvironment. That is, element characteristics 301-305, 321, 322, 342 canchange over time. Characteristics of element 301-305 can include, but isnot limited to, element settings (e.g., image quality settings, depth offield, aperture), element position, element state (e.g., device on,device off), and the like. Characteristics of element 321, 322 caninclude, but is not limited to, position, intensity, color, quantity,and the like. Characteristics of element 342 can include, but is notlimited to, surface color, refractive index, and the like.

FIG. 4 shows a diagram of a system 400 for determining a true color ofsemi-reflective surface in accordance with an embodiment of theinventive arrangements disclosed herein. In system 400, anomnidirectional camera 410, an end-user device 430, and an imagingserver 450 are communicatively linked via network 402.

In system 400, the omnidirectional camera 410, having multiple fixeddirectional lenses 412, can be replaced with a set of distinct cameras301-305 as shown by FIG. 3 or can be replaced with a stereo camera. Theimportant characteristics are that multiple different lenses exist thatcapture images (404) of a common point, where these images havedifferent recorded color values for the common point. Thus, there needsto be a significant angle difference between the lenses, which resultsin different colors at a common point on a surface on an object (due tochanges caused by the specular contribution).

Thus, the omnidirectional camera 410 (or set of distinct cameras inanother embodiment) create set of images, where the images show a commonpoint from a semi-reflective surface, where the point has differ colorvalues in the set of images. The image server 450 determines a truecolor for the point in a manner consistent with the process of FIG. 1.The end-user device 430 is able to display images in true color, afterthese images have been processed by the image server 450.Omnidirectional camera 410 conveys images 404 over network 402 toimaging server 450 for processing. The processed results of the images404 is a set of one or more resulting images 406 having true color. Enduser device 430 receives and displays these processed images 406.

The functional division of components and processes shown in system 400is for one embodiment only, and other arrangements are contemplated. Forexample, in one embodiment, the image server 450 can be adapted/modifiedto perform the stitching functions illustrated as being performed by theomnidirectional camera 410. In another example, the end-user device 430can perform one or more functions shown as being performed by theimaging server 450.

In system 400, the omnidirectional camera 410 can include two or morefixed directional lenses 412, a stitching lens 414, a processor 416, astorage space 418, a transceiver 420, an audio transducer 422,software/firmware 424, and/or other such components. The lenses 412 cantogether cover an aggregate arc greater than any individual lens alone.Because the lenses 412 are in a fixed position, a surrounding arc ofenvironmental imagery is continuously being captured, which permits anend user to adjust viewing within this arc. The stitching function maybe separate from any determinations of true color (especially if alphablending techniques are used). Further, calculations for determiningtrue color, as detailed herein, do require significant differences oflenses angles, which may require the lenses be an array of lensespositioned a significant difference from one another (such as imagescaptured by two different cameras). The requisite significant angle willdepend on the sensitive and accuracy of the camera lenses andenvironmental conditions, as is evident to one of ordinary skill.

Each lens 412 can be an assembly of components able to capture makeimages of objects electronically. Each lens 412 can have acharacteristic focal length, aperture, sensor, image stabilizer, andlens mount.

The stitching lens 414 can be a wide lens angled upwards, which is usedto capture imagery for a wide region in order to stich component imagestogether. The stitching lens 412 can overlap with areas captured by eachof the fixed directional lenses 412. Fidelity (pixel density) of thestitching lens 414 can be significantly less than fidelity of the fixeddirectional lenses 412.

The end user device 430 can be a device able to display one or moreimages within a scene 438 of a graphical user interface 437. Thus, enduser device 430 can be a computer, a notebook, a tablet, a smart phone,a kiosk, an entertainment system, a media player, a television, andother consumer electronic devices. End user device 430 can include aprocessor 432, a storage space 434, a transceiver 435, a display 436, auser interface 437, and/or other such components. Scene 438 can be atwo, three, or four dimensional computer generated scene includingcontent of image 406. Scene 438 can be a static or dynamic scene.

The imaging server 450 can be a set of one or more machines that performimaging functions. The imaging server can include one or moreprocessors, storage spaces, a transceiver, program code able to beexecuted by the processors, and/or other such components. Imaging server450 can include, but is not limited to, color engine 460 and anenvironment component 470.

Color engine 320 is able to determine/calculate a true color for pointspresented in multiple images, in a process consistent with that detailedin FIG. 1. Color engine 320 can include, but is not limited to, imagecollector 462, a color calculator 464, and a set of configurablesettings 466. Color engine 430 may utilize images captured by two ormore different omnidirectional cameras to determine a true color forpoints on a surface of an object.

Image collector 462 ensures that a single common point is identifiedacross a set of images 404. Thus, the image collector 462 maps pointscaptured in images across a set of images 404. The image collector 462can select a subset of images 404 from which to determine true color inone embodiment. During selections, image collector 462 may considerimage quality properties, such as: sharpness, noise, dynamic range,contrast, color accuracy, distortion, vignetting, exposure accuracy,lateral chromatic aberration, lens flares, color moiré, artifacts, andthe like. In one embodiment, images of the image set 404 with low detailcan be rejected and images with high detail can be utilized.

The color calculator 464 determines a true color of a set of one or morepoints and uses that true color when creating image 406. The true coloris based on a combination of diffuse component and ambient component oflight, which are insensitive to camera/lens angle.

Settings 466 can be one or more rules for establishing the behavior ofsystem 400, server 450, camera 410, and/or device 430. Settings 466 maypermit a set of processing rules to be established for differentsituations. Further, settings 466 can establish user-specific, camera410 specific, and/or context specific processing parameters.

The environment component 470 defines variables of significance for truecolor determinations. The environment component 470 defines surface 472characteristics, a set of surface points 474. Light source 476 position,intensity, color, and other parameters 478 are computed and defined byenvironment 470 component 470. Likewise, a position, angle, and imagingcharacteristics (parameters 482) of the lenses can be defined as cameras480 of the environment model.

The images 404, 406 can be digital artifacts that specify image details,so that a machine is able to display and/or print an image using thespecified attributes encoded within the digital artifacts. The images404, 406 can conform to a number of raw image formats. Formats need notbe the same from image-to-image, and may vary. Image 312 format canconform to a, Joint Photographic Experts Group (JPEG), Portable NetworkGraphic (PNG), and/or other such standards or derivatives thereof. Itshould be appreciated that image 312 can be a portion of a video contentsuch as a Moving Picture Experts Group (MPEG), Audio Video Interleave(AVI), and other video formats.

Network 402 can include any hardware/software/and firmware necessary toconvey data encoded within carrier waves. Data can be contained withinanalog or digital signals and conveyed though data or voice channels.Network 402 can include local components and data pathways necessary forcommunications to be exchanged among computing device components andbetween integrated device components and peripheral devices. Network 402can also include network equipment, such as routers, data lines, hubs,and intermediary servers which together form a data network, such as theInternet. Network 402 can also include circuit-based communicationcomponents and mobile communication components, such as telephonyswitches, modems, cellular communication towers, and the like. Network402 can include line based and/or wireless communication pathways.

Storage spaces 418, 423, and/or a storage space of server 450 representdata stores able to be physically implemented within any type ofhardware including, but not limited to, a magnetic disk, an opticaldisk, a semiconductor memory, a digitally encoded plastic memory, aholographic memory, or any other recording medium. Each of the storagespaces can be a non-transitory storage medium, which excludes carrierwaves, signal mediums, and other forms of energy. Storage spaces can bea stand-alone storage unit as well as a storage unit formed from aplurality of physical devices. Additionally, data can be stored withinstorage spaces in a variety of manners. For example, data can be storedwithin a database structure or can be stored within one or more files ofa file storage system, where each file may or may not be indexed fordata searching purposes. Further, storage spaces can utilize one or moreencryption mechanisms to protect stored data from unauthorized access.

The flowchart and block diagrams in the FIGS. 1-5 illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

What is claimed is:
 1. A method comprising: a computing devicedetermining a common point of an object present in the plurality ofimages, which has a color value that differs among a plurality of imagesshowing the object; the computing device mathematically expressing colorvalue of the common point as a set of equations, one of the equationsfor each of the plurality of images; the computing device making anambient contribution and a diffuse contribution for color as definedwithin the set of equations approximate equal across the set ofequations; the computing device quantifying a specular contribution atthe common point for at least a subset of the plurality of images; thecomputing device substituting the quantified specular contribution valueinto a corresponding variable and solving the simulations equations fora value of the ambient and diffuse contribution at the point; and thecomputing device establishing a color of the point as the ambient anddiffuse contribution, wherein the established color is a value differentfrom any value recorded for the point within any of the images.
 2. Themethod of claim 1, further comprising: the computing device creating anew image based on the plurality of images that uses the establishedcolor of the point.
 3. The method of claim 1, further comprising: thecomputing device modifying one of the plurality of images so that theone image as modified uses the established color of the point.
 4. Themethod of claim 1, wherein each of the equations comprises a recordedcolor value for the point, an ambient contribution, a diffusecontribution, and a specular contribution.
 5. The method of claim 1,wherein the established color represents the true color at the point. 6.The method of claim 1, wherein differences in color in the plurality ofimages at the point are based in part upon an angle of a set of lensesused to capture the image relative to the object being captured.
 7. Themethod of claim 1, further comprising: the computing device calculatinga position, color, and intensity of each light source that illuminatesthe object in the plurality of images.
 8. A computer program productcomprising a non-transitory computer readable storage medium havingcomputer usable program code embodied therewith, the computer usableprogram code comprising: computer usable program code stored in anon-transitory storage medium, if said computer usable program code isexecuted by a processor it is operable to determine a common point of anobject present in a plurality of images, which has a color value thatdiffers among the images; computer usable program code stored in anon-transitory storage medium, if said computer usable program code isexecuted by a processor it is operable to mathematically express coloras a set of equations, one of the equations for each of the plurality ofimages, wherein the equations comprises a recorded color value for thepoint, an ambient contribution, a diffuse contribution, and a specularcontribution; computer usable program code stored in a non-transitorystorage medium, if said computer usable program code is executed by aprocessor it is operable to quantify a specular contribution at thecommon point for at least a subset of the plurality of images; computerusable program code stored in a non-transitory storage medium, if saidcomputer usable program code is executed by a processor it is operableto simultaneously solve the set of equations for a true color value ofan ambient and diffuse contribution at the point; and computer usableprogram code stored in a non-transitory storage medium, if said computerusable program code is executed by a processor it is operable toestablish true color of the point as the ambient and diffusecontribution at the point using results of the simultaneous solving ofthe set of equations, wherein the established true color is a valuedifferent from any value recorded for the point within any of theimages.
 9. The computer program product of claim 8, further comprising:computer usable program code stored in a non-transitory storage medium,if said computer usable program code is executed by a processor it isoperable to create an image from a set of the plurality of images or tomodify at least one of the plurality of images to specify the true coloras a color for the point in the created or modified image.
 10. Thecomputer program product of claim 8, further comprising: computer usableprogram code stored in a non-transitory storage medium, if said computerusable program code is executed by a processor it is operable tocalculate a position, color, and intensity of each of a plurality ofdifferent light sources that illuminates the object in the plurality ofimages; and computer usable program code stored in a non-transitorystorage medium, if said computer usable program code is executed by aprocessor it is operable to quantify the specular contribution for eachimage utilizing the position, color, and intensity of each of theplurality of different light sources.
 11. The computer program productof claim 8, further comprising: computer usable program code stored in anon-transitory storage medium, if said computer usable program code isexecuted by a processor it is operable to quantify the specularcontribution for each image by determining a sum of a specularcontribution of each of a plurality of different light sources thatilluminates the object in the plurality of images.
 12. The computerprogram product of claim 8, wherein the true color is a color expressedwithin the plurality of images for the point by contributions that areinsensitive to camera angle, wherein the specular contribution that isremoved when determining the true color is sensitive to camera anglecausing a recorded color value to change depending on a camera angle.13. The computer program product of claim 8, wherein the surface of theobject at the point is a semi-reflective surface.
 14. The computerprogram product of claim 8, wherein a set of lenses used to capture theplanarity of images are lenses of a single camera, which capture theimages of the object at within one second of each other.
 15. Thecomputer program product of claim 8, wherein a set of lenses used tocapture the planarity of images are lenses of a plurality of differentcameras, which result in the common point being captured atsignificantly different angles.
 16. A system comprising: one or moreprocessors; one or more non-transitory storage mediums storing programinstructions that the one or more processors execute; computer usableprogram code comprising at least a portion of the program instructionsto determine a common point of an object present in the plurality ofimages, which has a color value that differs among a plurality of imagesshowing the object; computer usable program code comprising at least aportion of the program instructions to mathematically express colorvalue of the common point as a set of equations, one of the equationsfor each of the plurality of images; computer usable program codecomprising at least a portion of the program instructions to make anambient contribution and a diffuse contribution for color as definedwithin the set of equations approximate equal across the set ofequations; computer usable program code comprising at least a portion ofthe program instructions to quantify a specular contribution at thecommon point for at least a subset of the plurality of images; computerusable program code comprising at least a portion of the programinstructions to simultaneously solve the set of equations for a truecolor value of an ambient and diffuse contribution at the point; andcomputer usable program code comprising at least a portion of theprogram instructions to establishing a color of the point as the ambientand diffuse contribution, wherein the established color is a valuedifferent from any value recorded for the point within any of theimages.
 17. The system of claim 16, further comprising: computer usableprogram code comprising at least a portion of the program instructionsto calculate a position, color, and intensity of each light source thatilluminates the object in the plurality of images; and computer usableprogram code comprising at least a portion of the program instructionsto quantify the specular contribution for each image utilizing theposition, color, and intensity of each light source.
 18. The system ofclaim 16, wherein the established color is a color expressed within theplurality of images for the point by contributions that are insensitiveto camera angle, wherein the specular contribution that is removed whendetermining the established color is sensitive to camera angle causing arecorded color value to change depending on a camera angle.
 19. Thesystem of claim 16, wherein the surface of the object at the point is asemi-reflective surface.
 20. The system of claim 16, wherein a set oflenses used to capture the plurality of images are lenses of a singlecamera, said lenses having different angles with respect to the object,which capture the images of the object at within one second of eachother.